Skip to content

net::ERR_ABORTED at "https://..." from a crawling program #9363

Closed as not planned
@Redskinsjo

Description

@Redskinsjo

Bug description

Steps to reproduce the problem:

  1. enclose a 'new Promise' whose body includes a 'page.goto()' then a 'page.evaluate()' inside a map that returns into a 'Promise.all()'.

const getResult = async () => await Promise.all(websites.map(site => {
const promise = new Promise(async (resolve,reject) => {
try {
await page.goto(site, {waitUntil: "networkidle2"})
const result = await page.evaluate(() => {
...
return result
})
resolve(result)
} catch (err) {
reject(err)
}
})
return promise
})

try {
console.log(await getResult());
} catch (err) {
console.log("OKOKOK", err.message);
}

// msg.text() is from a console 'on' event

Puppeteer version

^19.3.0

Node.js version

14.17.6

npm version

yarn 1.22.19

What operating system are you seeing the problem on?

macOS

Configuration file

no config file, simple node server with js scripts

Relevant log output

msg.text() Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.
msg.text() jQuery.Deferred exception: $(...).tooltip is not a function TypeError: $(...).tooltip is not a function
    at HTMLDocument.<anonymous> (https://www.{site}/sites/default/files/js/js_gQZflpXLjI8oiEOVh9W7qG81ILOXaEg7B8xYt5OFDqQ.js:8924:25)
    at e (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30038)
    at t (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30340) 
msg.text() hello ok
OKOKOK net::ERR_ABORTED at https://{site}.com
msg.text() Error with Permissions-Policy header: Origin trial controlled feature not enabled: 'interest-cohort'.
msg.text() fancyBox already initialized
msg.text() jQuery.Deferred exception: $(...).tooltip is not a function TypeError: $(...).tooltip is not a function
    at HTMLDocument.<anonymous> (https://www.{site}.com/sites/default/files/js/js_5aKlLkXJ30Pe47l8D-WslgYctBeM3RPXOkR-UpKXcJ8.js:2845:25)
    at e (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30038)
    at t (https://www.{site}.com/sites/default/files/js/js_hTpwK1S7vrS1qPQTgcre974phYT7ajx29jtfj8BBGSE.js:2:30340) 
✨  Done in 7.00s.

Activity

Redskinsjo

Redskinsjo commented on Dec 6, 2022

@Redskinsjo
Author

seems to me that it is similar to #8246 with a newest puppeteer version

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
Collaborator

@Redskinsjo could you please provide a minimal example that we can run? (https://stackoverflow.com/help/minimal-reproducible-example)

Redskinsjo

Redskinsjo commented on Dec 6, 2022

@Redskinsjo
Author

sure. stackblitz and codesandbox have issues to connect to chromium, even when I install chromium locally.
anyway, please find the reproduction here :)

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
Collaborator

@Redskinsjo so you are sending concurrent navigation requests to the same page and the first navigation gets aborted by the second one. Why do you think it's a bug in Puppeteer?

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
Collaborator

You'd probably want to rewrite your code code as follows:

(async () => {
  const browser = await puppeteer.launch({
    headless: false,
    executablePath: executablePath(),
  });

  const page = await browser.newPage();

  page.on("console", async (msg) => {
    console.log("msg.text()", msg.text());
  });

  const getResult = async (jobSectors) => {
    const results = [];

    for (const i of [1, 2]) {
      await page.goto("https://www.google.com/", {
        waitUntil: "networkidle2",
      });
      const result = await page.evaluate((i) => {
        return i;
      }, i);
      results.push(result);
    }

    return results;
  };

  console.log(await getResult());

  await browser.close();
})();
Redskinsjo

Redskinsjo commented on Dec 6, 2022

@Redskinsjo
Author

i had to change the URL. but I am basically crawling the website and requests are sent to different URLs. Can't Puppeteer handle multiple page.goto(url) at the same time ?

OrKoN

OrKoN commented on Dec 6, 2022

@OrKoN
Collaborator

you should create multiple pages in that case (via browser.newPage).

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

      Participants

      @OrKoN@Redskinsjo

      Issue actions

        net::ERR_ABORTED at "https://..." from a crawling program · Issue #9363 · puppeteer/puppeteer